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(54) METHOD FOR EXTRACTING CORRESPONDING AREA OF IMAGE, METHOD AND 
DEVICE FOR COMPOSITING IMAGE, IMAGE COMPOSITING PROGRAM, AND 
COMPUTER-READABLE RECORDING MEDIUM RECORDING IMAGE COMPOSITING 
PROGRAM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To accurately extract a 
corresponding area from two images without being 
influenced by a change in a luminance level. 
SOLUTION: Secondary differential images are found out 
by secondarily differentiating two original images, and 
while changing an overlapped area between the two 
secondary differential images, the number of edges of at 
least one of the secondary differential images on the 
overlapped area is calculated. Then, an error map is 
found out based on an error between the secondary 
differential images on the overlapped area and a 
corresponding area between the two original images is 
extracted based on the number of edges calculated and 
the error map found out. 
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[11*31 1 ] 2 «oj((i**n*ns 2 ytrnftisfz 2 

»^a^©X 5/ ytSIfilTS £ £ "b iC. life; D ffi«tc 
WK^ISv y i 4fife 0 «tt£JflHJ 

y y©«/h«K:i5tt* 2 2 

m>am6] mmx.y^mmmMmmmz$>i>^.x. 

v y y its 2 xmiNlL&Miziitz* 2 #cft 
#B^©S&D«*££»ttiU feM&DlgWcM/fc-r-S 30 

raws* 7] miMimm£Wi2mwm"tti^n&2& 

5 1 y y. tli#ai:I^^T« 1 ECHilftSfcttJfS 2 KB 

[it*JS8] HdBffilXfy^t tfffa#MH«*2 40 
&Vt*'J>te< £fe— #©2&«#B^©X>yy&£|£{ij 

1. 1 *> k, sj& ofH*tc*stt*w 2 x.m.ftmmmo 
mm\zm-?^xmm-? yy&^tbzx^y?, mm^-y 
zs&hwamm-? y y £ ic»y ^twmb#?w©mjs« 

Kfffi»|gVyy75t*/hffl*£*aa:Ofil«*tttH 50 
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[HtsRJS 1 0 ] tfrlBx v 5?ft«*Bf3tttfil±T * 0 . fl 

y y©«/hfitc*JttS 2 &«$H6tf*«;*:£&S 2 
B»©a&Dffi«£»fflU M^Offl^icMiST^^ 
SBI<fe©««&]SBH»W©*f««« £ Lxaait 4 it* 

js 8 ia«©(B»-&^*Sio 

[it** l l ] (HOTx-y i*ffoWMif(PilfcrtK:**£ 

Hi tar ^11*31 8 mm^mm^m^. 
m&m 1 2 ] mm^y >?*dJi&»}jsiiM8irt^**^ 
x>y^}c*fbT0fs©«^±T*o, mmm 

Slv y y©S/jN«tcistt * 2 2 
8 IH*©H#B-^^S5o 

[1**11 i3] jg 1 Mimmtm 2 mm^n^tiz 2 
'&r&ttvfz2'AWftmm$:)k#>. m2^m»mmizm-zs 

jS«J*(c*3tt-5 ia5igiiI«P B 1©M«i!C*ff UJt^I 2 ^ 
9:. KW«a:fcl6tJv»T*lfni<|i*fcJ4IB2i(H** 

sadbTW«B«*^rirr*fB 3 

Si. 

[W*« 1 4 ] iuiaig 1 tuia#Miii^^ 2 ^ 

mv>mt£ 0 mm a s > 0 «« jc * it * 
^< £ *© 2 ^c«^H^©x-y ^sscsncm-r* £ 

£ *> ic, Sfj 0««ic*3^4M 2 ^^B#M©^(C 

itv v -ft izm-D'^xmmmm^cDMmmm^mmr 
5#a*#x.fc»*ii 1 3 ia*©a«^SBo 

[«*JS 1 5 ] ffr fHx -7 -^**i3f3£#£t±-C* 0 , J. 
UfiRlVy y **»/h|t * £ s fi^ 0 s a a 

l, ^fi^i3««{c*rjsT4MH#©«j«*Ma^m© 

*tJSM^£LTtttblT4f»*3Sl 4Ei0HfMS 
Bo 

[»*3S 1 6 ] stjiax y 5?*«j»f3e«fit±-e* 0 , a 

mmcoMTiDmm^mmv. mm^^mmzM^-r^^- 

mi 4fa*©a«^ggo 
[«*jsi7] mw^y*j®L*mnmmmH\z$>z>± 
x v vmzn\^xi%ii_<nm^Atx$> 0 < ffriaig 

«* » iST 4 Hiift ©«« S MB#fW©*f mm®. £ L 

T«iw-r*»*«i 4ta«©a«^sSo 
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m&m is] mm^y ^mmmmmmm \z&z>± 

M^y7<vm'Nmz£ilfZ>2'Affi.ft»i!)m±£tii2>2X. 

^xim^m^m^Mj&m^mm-r^ i x. 

^M*S««lC^W-5WMlB#W©M«^:*#aT«)^2 10 
Xt^X, KM^iClcXc5"^TlglMH»Sfcttm2M 
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fcWMfc^lS 7" d X 5- A * IE® b n > fcf a. - 9 WtfrM 0 

matm 2 0 ] nei ixf7 x«. tfria^s-Mia^s- 

<D^&zM-3^TWA"? v ^**S^f7 7, WIEx 20 
a«£X D X5 A €f5S b fc rj > If 0 ~SSmz 

MJ£fPJ^£bTfflHjT£f»*>S2 0 3E*OH«^yp 

£IH® b 3 > fcf 3. - * St^ift 0 HJtg&lHIME 
fto 30 

im$tm 2 2 ] tuiax ? vmtmmm,±.-i:$> o , k 

y XcDffi/MBtiC&W* 2 2 5*C«fJ- 

JS2 0K*t©iH<^j£XD^7A*fE«bfc3>tf;i — 

im&m 2 3 1 musx^ ^wctmmmmmmz$>^>± 
x.y ismzttVTffife<Dm'&EJL±-v&K) . wian 
igv3/X#«/h«£<t*s&0«tt*Jftfflu tiara 0 40 

l 3 > t° a - ^ a*-® o BinefciEefiEtt. 
[i«2 4] tft»!xyj?*3&iStr»!JSBf<fti*g{c»*^ 
x-7>^icMbTj^©fd^JXtT$>D. mifflm 
M^yy'<nM'^mz&\l&2#zffiiffl&tfM*ittzZ>2X. 

m^s 2 4 ta*© a«! ^ 7" p x 5 a * tae b & 3 > tf 



[«*3S2 5] n»ti2iii«n ; en*2 
^^Lfc2^^iH«&5Rs6> m2^M.^-mmzm^ 
v>Titfiesiii»w©**jsfs**iawT*^ 1 x, 

Xf-yy, KM«SClcS-rf^T» 1 JSW«£ fcttlg 2 JSC 
Bt^giLTMMPit^Mt^I 3Xf7 X£#fx. 

[§£fl©ffiMffl&§iM 

[0 0 0 1] 

m^m wi^Tm b*£j&&k> m^fe-futf 

7A. Bi^}&XnX5A£iE^b;fc3>k°:x-^M^ 
[0 0 0 2] 

< M&M <D H V >jB# £ Si a g Slfc & V i J; 5 f£ 
[0 0 0 3] /V 5 VH#*Mt5fttBi iTtt. t> 

«) Sfrs2ft^a-y->*i«L,feft < ^»#tffi©ffi 

[0 0 04] fU, ^CD2^CDIH«!C*3tt^*jl(?5« 
T. — SSiCSSDffi (Sum of Square Di f ference) £ H 
[0 0 0 5] ifflSSDSll £"T, 3— +f{Cj:oTJS 

5£$tifc 2 ftoiH«*/i v 3 >rt®^tij ^ctt*-JA^fc 

^> 2ft©Bf^-f-n^ntC-3ViT, IBtfeKBW 
=tO*^OffiViiH»J 1 , J2^T5. d©2fe 
©BiJ 1, J 2 Ol&0S« co (it-YX : MXN) 

i^it^c i \z^^o\zmm&tz i oco2mwM (e) 
*ffl^T*s6?>n^o T^cfc*., a#raco#»a (d) 

(d) tOiiCl&Oiiw^JSStiL 
[0 0 0 6] 



£"(d) 



I ffl C/i(^)"/2(^d)) : 
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s £ t #«t £ & v>ata** s . 

[0 0 0 8] *jfc. fc^&fcgXyS^B^fcVMMfrr 
«, fcfJfcfiStt**— iScbTV^<Tfc S S DOtf-#IS^ 
tff& < f£ 0 , M^©atU*Ko TLi'5 -5 „ 

[0 0 0 9] -tUT, *5PJte. mm.WKDmt'PX- 

v*j(D&fz^m&<D&mzMmziriz>z.t.u< 2#c©b io 

[0 0 10] 

«a«ft&3fc«>, to 2 ^^-B^fcs-^TiCB^ffl 

So 

[0 0 11] *56M©iaiSOJ*lSffi«ttW*ffi 
fcCDffi^(C*3W^d>^£< *©2^«^IlI#COX-y 

mmm^mmizm^xmm-? >y y^a>, x -y ytt 

feCDT^So 30 
[0 0 12] H^Wf'tt, x-y >?^0f^fitiy.Jt-rfe 

W©MJS««£L.TrtffiT£fc©T&-5<, ^ntciO. 
Ttttti £fnT\ j^«J:DXy^#©#^£fc9«*© 

* a> s « t> wis ©4> <^fi& o tLii 

[0 0 13] xy5^fca*BrJ£*£U: 
TSD, J=L^>> RSVyT'S'i'hfiSit). St?, KM 40 
v -y ycDS/h«{c*$ftS 2 j*c«4M»*»»*tft:* 2 XM. 

^>#MB«©«^*MB#ffl©*fJs«^ch 

[0 0 14] JWfcOTlCte, X-y>?ic^MB»rt(C 



^tLTfamr-5fe©T-$)«.o ^nsc^o, MB&fc*t 

[0 0 15] Sfc, ftflEfcJfctt, Zy5?ft««W«|«Jl: 
7"©«/Mfitc£;W-£ 2 *liM8*5i^:t&5 2 

CN-r^#MB#©««^MB^m©*t«««t Lxatis 

[0016] *fS0j©B*^73S;tt, mimmmzim 
2 mm#. j tti*eti& 2 ^w^vtz. 2 ^m.^-mm^tt>. 

^© 2 ^#Bf&(c*^TMBItl!f]©#J&ffi«£»ffl 
S«it^*m-r-S)»2X-r-y7", ^©w#5tfc«-o*^T 

is 1 mm^tcitm 2 iBt^tiLxssBt^M 

[0 0 17] ilXf77"tt. #MB#£ 2 
2^®^B»*#Wr^X5 : --y7\ ffi2Jfc«$HB*fclffl© 

< <hfc— yj©2*«^B#©X-y> s »:*^ttJ'r Siife 
(C. fife 9fct«£*5«-*ffl 2 ^7>B^W©^M(^-^ 
HT^iV -y ^^id-SX^y T7\ X-y v^t^MV-y 

[0 0 18] M^mizU, Xy >>»c*«3f3£tt£t±-C* 

U •€-©S*D««5c^-r^ilB«©««*MB»W 

[0 0 19] Sfc, ft#Wi:H, Xy^ffc&*Bfj£ffiJ£l_k 
T&D, R^v-yy"^S/Hjt*t 0> fio, MM 

v >y ^co«/htttci5 2 2 ^cft 

^»#;jgB«©««*iSB#ra©*fjs«iti bxttta-rs 

fe©T$.So 

[0 0 2 0] X-y^m^KB#rtic: 
ii7 -y ^*/J^^ £ 0 ««*»i±J L, ^©a 

[0 0 2 1] M^WtCtt. X-y^ic^MB^rttC 

■y^i/htStt). KZ3, mm-?y7"0)M'Nmzls!,? 
•5 2 ^W^fi^ft^cttt* 2 &«#lH|fe©S;&: 0«tt£ 
fflWb, ^©«^D««(c*flS-r-5^MB^©ffl«^S 
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[0022] ^mcomm^mmi-i, mimmmtm, 

[0 0 2 3] ilfSH #JgiH#^2^«^Lfc2^ 
[0 0 2 4] JMfcWicW:, X^>?m^J5f^fiUy.JiT& 

©MJKMWt bxttffl-r-S) fc©-c&-s. 

[0 0 2 5] Sfc, Xy>?»c*«3f)£tt£t± 20 

V y 3/©«/h«{C:fc W£ 2 t&S 2 ^® 

^B#©fi&D»*£:tttriU -tOfi&DftltttettJS-r 

[0 0 2 6] Sfc, JWfcWSCfct. X>y^^®ia#rttC 
&^x-.y ^ifcJcMLTfllT^©f^£U:T-& 0 . J.'O. 
ii? y y #*/J\flt * <h -5 S& D 0WE * tB b . * Ol 

tbTttf±i-r'5t)©TS'5o 30 
[0 0 2 7] Jl^WSCtt, l7yS«i^l: 

[0028] *«woiB<ft^fiRyp^5A*aaeufc3 

2 Rlft^n^tl* 2 iW"4#Lfc 2 

2 ^®5j-ja#!'S-3"^TjgCH#|IB«^««Srttai 40 

[0 0 2 9] ilXT'y^B, ^MBi*2«^lfc 

mfco mutt. & Mit s itfca* e » a& o w« tc*; ft -a 

< <tt)-*©2Jfc«^iB«©X>v^**J!£lll-r*ttfe 
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I^TSIIgV y y *5J<S6 5 X y y, X y s?& t. IfV >y 

[0 0 3 0] Xy5?»*«Bf3e#[JB[±T?& 
U *ofifeOfR«{c^ic:-r*SiiBI»©fs«sSBIIfelffl 

©*fJ«SW<i: LTttttj-TS &©T&-S. 
[0 0 3 1] IWfcMJCtt. Xy5?ftafl*8f£*£Ui 

[0 0 3 2] JWTOiCtt, Xv^«C^JMHI#rtiC 

aba^xy^Sfctc^LTBfjeoSd^-hT?*?). fio. 

it7 v y&M'hm stsaao w «t * a m l . ^©s 

[0 0 3 3] Sfc. AttWfCtt. Xy3^k*«JWmtrtK: 
**#Xy3?SkK«LTBfS©«l^±-C*0, 

[0 0 3 4] 

ci] ti)vy a— mjs.Tomz'D^T commo 

[0 0 3 5] *y°<r4 M7D- Jt5fefr£sW:, fcf^Sy 

y^ p-*a»WKtnf-r*iejBMJfi5t*i&*i 

tL, tlJVy ©tf-^^j^tts Lucas-Kanade 

&3BHffi (C £ * ^- ^ ^ * ;W 7 □ *Su IS t L T 

V^. ^BEStbTLucas-Kanadefe^ffiVi 

[0 0 3 6] Sfc< ^-y"^^ tuuya—m&7jmz^ 
Ttt, pgg*jfiftbfcLucas-Kanade)*{C«t-5^-^-Y 

p— »^fe©#®Pgic*3^T#6tife^-y^^ 
7D-#, Bmmmiz^-DTWZtznz, j^t. ^nic 

0Lucas-Kanadefe 

Lucas-Kanade7£©fMiffl{-'^V5Tte> Xtt : B.Lucas and 
T.Kanade,"An Iterative Image Registration Techniq 
ue with an Application to Stereo Vision, "In Sevent 
h International Joint Conference on Artificial Int 
elligence(IJCAI-81) t pp. 674-979, 198lSr#M©^ 
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[0 0 3 7] SZITIZ, Lucas-Kanadeffi©#£g£jyw-<-g>o 
[0 0 3 8] &Z>mmt <Z>iBlfcSE«P=<x,y) (DW&Lrt* 
->I(x,y, Otfi. $>Z>fflL'^mM'&(6 DtCJMfKx+Sx 
, y + <5y ) ic, f©MM§-S!:«^S:Sii 



[0 0 3 9] 
[ft 2] 



+ 



= 0 



2^pHftT:t:7"5V {v =( <5x /8 \ , 

<5y /<5t ) =(u,v)} Sfff-ff-T-SiCtt, *£Q/^;* — 

Lucas iKanade(^tii) tt, |W|-tl#:»^J?f««T«, R 

[0040] mnk±<ommm.i&cofHT\ ^r-ff- 

Io (p) = I(x,y, t), I, (p + v) = I(x +u , y +v , t + 
5 t) 

t.m^?kSbZ>£. ^itft3T-^«T^«>o 
[0 0 4 1] 
[ft 3] 

£ = Z[/,(P + v)-/„(p)] 2 

a> 



30 



v* + , =v t + 



2>*) 2 



10 



[0 0 4 2] 
[ft4] 

h (P + v) = /, (p) + g(p) v 

g(p)tt, I, (p) ffl-M»T*5. 
[0 0 4 3] KHE^*/Jx(C^-S©tt, E © v lZM~^-%> 

10 [0 0 4 4] 
[ft5] 

= Z 2 s(p)I7.(p) + g(P)v - / 0 (p)] 

20 [0 0 4 5] 
[fc6] 



v « 



S^(p) 2 



[0 0 4 6] 
[*7] 



/* =/ 1 ( P +v k )^ ^(p+v k ),/ 0 =/ 0 (p)-efcS o 



[0 0 4 7] HI (d) t£JK]94fe&, 0 1 (c) ttBl 
(d) oAlMlfcJ:OjRlkdeoffiV)Bft«. 01 (b) tt 

HI (c) CDffi^#*iffl#J;D^S^<g^jB#*, H 

1 (a) Ml (b) 

(fc£x.fcf 1 3x l 3®*) (DnyT-mz 

*a6^ti^o HI (a) -HI (d) Kl&iAT. SH 

1 OCD/^yf^lW^o 50 



[0 0 4 8] HI (a) coHfft (l^fllcoH^) . Hi 
(b) <DWM (K«2<BiB#) , HI (c) OPHft (Pit 

13(7)11) MffBi. (d) (Dmm (mm4(Dmm 
cd «b#t apg w tc * y~T~ ^f})vyu — s na 0 h 

1 (a) -HI (d) fc43Vvc3eBH4, At 

(3) »3B«iatCckS*t^ 
Lucas-KanadeffiCDS^CD^^^, iiBM6*3WMite*«f 
O^^TS-^o Tomasi £Kanade£te:, &Z>vMM.(Dj&Wf*I 

bfc(C.Tomasi and Kanade, " Shape and Motion from Im 
age Streams: aFac tor izat ion method-Part 3 Detectio 
n and Tracking of Point Features, " CMU-CS-9 1 -1 32, C 
arnegie Mellon University, 1991. ) 0 
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[0049] 3b&Mmmmto<Dmm • 7K¥*[fi]«>®#© 

2**S3»S^05*C^»8CD2 X 2©«*fcfT5IJGj5^. 

[0 0 5 0] 
Bfc8] 

G = 2]g(P)g(P) r 

pew 

§o fcT> /hSVi#0ltttAmint, jgH^<Z>« 

[0 0 5 1] 
[«C9] 




v^«Tt>, 4-:/^ #;i^y d-^s^o £ 

[0 0 5 2] 0 2 fcyp — ^ hJKDBfaBaa^fil^* 
^to 30 
[0 0 5 3] ^D— ^ h;KD««ft<OV^y 

[0054] s-r. #5nfc7D-sus^»jait 

[0 0 5 5] 2«®#tcd^;1/^^d>?-s©:^ 

;i/u (i, j ) ti, J £<D4i[£&<D7n—'<t? b)]/&*>fg 40 
(B*r»cJKUT»*#»SffoTJfca*l Oco^pfcgt 

jrtr**. 

[0 0 5 6] 
[»1 0] 

/?,4f y a 

(p,ff) = (0,l),(0,-l),(-l > 0),(l,0) 

M 50 
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iz^XfrOo ft*, afe^ftt^i, j(D7D-^^hJl/ 

[0 0 5 7] 03 (a) tt, LTL 

(b) Hi^Otyr^ *^7D-§^LT^§o El 
3 (a) fciswr, 5gWttb*v^»afcJ:oTflWII« 

[2] B^^Sficoi^HJ 
0414, ***<0»JBfc*^SBI«^S«O*ltfc&^ 

[0 0 5 8] /\°— y^-;m>fcf ^ i oicte* fVx 

2 1 , 7^7 2 2 43 J:tf4=~#— F 2 3 HmHtS 

nxv^So a°— y^-;i/3>tfa.— # i cpui 

1. >C^U12, A-Ff>fXi7l3, CD-ROM0 
ct 5 & U A — )V^)]/^r-i X 2 <D F A* 1 4 ATti 

[0 0 5 9] A— F^-fX^ 1 3Ctt, OS (rt^U— 
tlfcCD-ROM2 OSrfflViT. A— F5^Xi? 1 3 fC 
H^trCO* 

[0 0 6 0] 0 5(1 /V7-7itMV7HA5g»it 
Ls65.tlfe«-&(CC PU 1 1 (Cdx-DTfffcn^^ty 5-^ 
HHft^fiJtf&a* MB £ ^ IT V i -5 , 

[0 0 6 1] CdTti, SJWOffiS[±, 2 tfcODjltff:*^- 

(I) St". 3.— !ffCcfc^T*S^$nfe2^CDB# (MSI 

mmRzfm mim ^/tu 1 2 tr^asti-s (x^ 

(id mz. 2^c<Dmmz-D^xMs^mmcomtiiMm^ 
fT^n-s (x-t-->^2) „ c©*rjts«*©jftm«iatt, 

[0 0 6 2] St, 2ftc7)a#I 1 .,«, i„„*tw?n 

fC^T. 2&tt#fflg£Jilxfc2;fc«#III4fc I, . I, 
««4lJE3ft«. * 2 ^c«4MH« I , . I.Ttt, JECnifel 

1) 

fix- 2^t^iii,, i i &mm<Dm&T?2®i£? 
^tc -Jj<d 2 'Am>mm 1 , *fm^© 2 #.$kftmm 1 , 
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I,, I 2 CDX>y^ (»iiltaSB$) ©Sal, a 
2**^7>hT^o fib. 2*«#H#I,<D»»*£ 

(d) t-rst. tcommm (d) js, M2^$^m# 

(d) »»C*S:0fWatco »-fX:MXN) fc^frtS 2 
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